Papers with low-bit quantization methods
MixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing low-bit quantization methods often exhibit severe performance degradation on complex reasoning tasks. |
| Approach: | They propose a plug-and-play method that uses a key channel's intrinsic quantization difficulty and relevance to the query to identify and preserve critical key channels that need higher precision. |
| Outcome: | Experiments on complex reasoning datasets show that the proposed method outperforms low-bit methods at a substantially reduced memory footprint. |
Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) require significant computational resources for deployment and use. |
| Approach: | They propose to use low-bit quantization methods to reduce memory footprint and increase inference rate to improve performance of Large Language Models. |
| Outcome: | The proposed methods can reduce the memory footprint and increase the inference rate of LLMs. |